Skip to content

Revert "Non blocking OlapTableSink" #3503

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
yangzhg wants to merge 1 commit into from
Closed

Revert "Non blocking OlapTableSink" #3503

yangzhg wants to merge 1 commit into from

Conversation

@yangzhg
Copy link
Member

@yangzhg yangzhg commented May 7, 2020 •
edited
Loading

this pr casue many localtests fail and in tablet_sink.cpp:225 requires return statment in non-void function

@morningman
Copy link
Contributor

morningman commented May 7, 2020 •
edited
Loading

Hi @vagetablechicken
Your PR #3143 may has some problem which cause BE in our CI environment crashed.
Some of core file shows like:

#0  tcmalloc::SLL_SetNext (n=0x0, t=0x26aca80 <vtable for doris::AlphaRowsetMeta+16>) at src/linked_list.h:49
#1  tcmalloc::CentralFreeList::FetchFromOneSpans (this=this@entry=0x40d6520 <tcmalloc::Static::central_cache_+21888>, N=N@entry=2, start=start@entry=0x7f912f9adda0, end=end@entry=0x7f912f9adda8) at src/central_freelist.cc:314
#2  0x0000000002539c0c in tcmalloc::CentralFreeList::FetchFromOneSpansSafe (this=0x40d6520 <tcmalloc::Static::central_cache_+21888>, N=2, start=0x7f912f9adda0, end=0x7f912f9adda8) at src/central_freelist.cc:282
#3  0x0000000002539d07 in tcmalloc::CentralFreeList::RemoveRange (this=0x40d6520 <tcmalloc::Static::central_cache_+21888>, start=start@entry=0x7f912f9adda0, end=end@entry=0x7f912f9adda8, N=2) at src/central_freelist.cc:264
#4  0x00000000025461d3 in tcmalloc::ThreadCache::FetchFromCentralCache (this=0x45dce80, cl=<optimized out>, byte_size=288, oom_handler=oom_handler@entry=0x267fa20 <tcmalloc::cpp_nothrow_oom(unsigned long)>) at src/thread_cache.cc:126
#5  0x0000000002681560 in tcmalloc::ThreadCache::Allocate (oom_handler=0x267fa20 <tcmalloc::cpp_nothrow_oom(unsigned long)>, cl=<optimized out>, size=<optimized out>, this=<optimized out>) at src/thread_cache.h:380
#6  malloc_fast_path<tcmalloc::cpp_nothrow_oom> (size=size@entry=264) at src/tcmalloc.cc:1855
#7  tc_new_nothrow (size=size@entry=264) at src/tcmalloc.cc:1981
#8  0x0000000000f305fb in doris::AlphaRowsetWriter::init (this=0x7029b80, rowset_writer_context=...) at /home/work/teamcity/agent/work/4304a7ff82e9a41d/core/be/src/olap/rowset/alpha_rowset_writer.cpp:53

OR

#0  SLL_Next (t=0x0) at src/linked_list.h:45
#1  SLL_PopRange (end=<synthetic pointer>, start=<synthetic pointer>, N=4, head=0x4600e00) at src/linked_list.h:88
#2  PopRange (end=<synthetic pointer>, start=<synthetic pointer>, N=4, this=0x4600e00) at src/thread_cache.h:238
#3  tcmalloc::ThreadCache::ReleaseToCentralCache (this=this@entry=0x4600bc0, src=src@entry=0x4600e00, cl=cl@entry=18, N=4) at src/thread_cache.cc:206
#4  0x0000000002546657 in tcmalloc::ThreadCache::Scavenge (this=0x4600bc0) at src/thread_cache.cc:224
#5  0x000000000103d4ac in doris::RowBatch::clear (this=0x8ef3340) at /home/work/teamcity/agent/work/4304a7ff82e9a41d/core/be/src/runtime/row_batch.cpp:293
#6  0x000000000103d8bd in clear (this=0x8ef3340) at /home/work/teamcity/agent/work/4304a7ff82e9a41d/core/be/src/common/object_pool.h:54
#7  doris::RowBatch::~RowBatch (this=0x8ef3340, __in_chrg=<optimized out>) at /home/work/teamcity/agent/work/4304a7ff82e9a41d/core/be/src/runtime/row_batch.cpp:301
#8  0x000000000103d8e1 in doris::RowBatch::~RowBatch (this=0x8ef3340, __in_chrg=<optimized out>) at /home/work/teamcity/agent/work/4304a7ff82e9a41d/core/be/src/runtime/row_batch.cpp:302
#9  0x000000000103571f in checked_delete<doris::RowBatch> (x=<optimized out>) at /home/work/teamcity/agent/work/4304a7ff82e9a41d/thirdparty/installed/include/boost/core/checked_delete.hpp:34
#10 ~scoped_ptr (this=<optimized out>, __in_chrg=<optimized out>) at /home/work/teamcity/agent/work/4304a7ff82e9a41d/thirdparty/installed/include/boost/smart_ptr/scoped_ptr.hpp:89
#11 reset (p=0x0, this=0x8ddbd70) at /home/work/teamcity/agent/work/4304a7ff82e9a41d/thirdparty/installed/include/boost/smart_ptr/scoped_ptr.hpp:95
#12 doris::PlanFragmentExecutor::close (this=this@entry=0x8ddbbf0) at /home/work/teamcity/agent/work/4304a7ff82e9a41d/core/be/src/runtime/plan_fragment_executor.cpp:538
#13 0x0000000000fcb9dd in doris::FragmentExecState::execute (this=0x8ddbb80) at /home/work/teamcity/agent/work/4304a7ff82e9a41d/core/be/src/runtime/fragment_mgr.cpp:213
#14 0x0000000000fcd476 in doris::FragmentMgr::exec_actual(std::shared_ptr<doris::FragmentExecState>, std::function<void (doris::PlanFragmentExecutor*)>) (this=0x4d51500, exec_state=..., cb=...)
    at /home/work/teamcity/agent/work/4304a7ff82e9a41d/core/be/src/runtime/fragment_mgr.cpp:399

Looks like something wrong with the memory usage. So we decide to revert this commit.
And could you please issue a new PR? And we will run our test on it, to find the bug.

And also, it has compile error.

@vagetablechicken
Copy link
Contributor

vagetablechicken commented May 7, 2020

OK, I will issue a new PR.
And the compile warnings shouldn't be ignored, we only add '-Werror' in debug mode. I think we need -Werror in release mode

@morningman
Copy link
Contributor

morningman commented May 7, 2020

This is fix by #3502. This PR will be closed

@morningman morningman closed this May 7, 2020
@yangzhg yangzhg deleted the revert-3143-non-blocking-sink branch May 8, 2020 02:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@yangzhg @morningman @vagetablechicken